KL-regularized reinforcement learning from expert demonstrations has proved successful in improving the sample efficiency of deep reinforcement learning algorithms, allowing them to be applied to challenging physical real-world tasks. However, we show that KL-regularized reinforcement learning with behavioral reference policies derived from expert demonstrations can suffer from pathological training dynamics that can lead to slow, unstable, and suboptimal online learning. We show empirically that the pathology occurs for commonly chosen behavioral policy classes and demonstrate its impact on sample efficiency and online policy performance. Finally, we show that the pathology can be remedied by non-parametric behavioral reference policies and that this allows KL-regularized reinforcement learning to significantly outperform state-of-the-art approaches on a variety of challenging locomotion and dexterous hand manipulation tasks.

现代对高斯工艺的近似适合“高数据”，其成本在观测值的数量中缩放，但在``宽数据''上表现不佳，在输入功能的数量方面缩小了很差。也就是说，随着输入功能的数量的增长，良好的预测性能需要汇总变量及其相关成本的数量才能快速增长。我们引入了一个内核，该内核允许汇总变量的数量通过输入功能的数量成倍增长，但在观测数和输入功能的数量中仅需要线性成本。通过引入B \'ezier Buttress来实现此缩放，该块允许在无需计算矩阵倒置或决定因素的情况下进行近似推断。我们表明，我们的内核与高斯流程回归中一些最常用的内核具有非常相似的相似之处，并从经验上证明了内核可以扩展到高大和宽的数据集的能力。

强化学习（RL）为可以在现实世界中自主互动的培训代理提供了潜力。但是，一个关键限制是RL算法对核心超参数和网络体系结构选择的脆弱性。此外，诸如不断发展的训练数据和增加的代理复杂性等非平稳性意味着不同的超参数和体系结构在不同的训练点上可能是最佳的。这激发了Autorl，这是一种试图自动化这些设计选择的方法。一类突出的Autorl方法是基于人群的培训（PBT），这在几个大型设置中导致了令人印象深刻的表现。在本文中，我们介绍了PBT式方法中的两项新创新。首先，我们采用基于信任区域的贝叶斯优化，从而可以全面覆盖高维混合参数搜索空间。其次，我们表明，使用世代相传，我们还可以在一次训练中共同学习体系结构和超参数。利用新的高度可行的Brax物理引擎，我们表明这些创新导致了巨大的性能增长，在即时学习整个配置的同时，大大优于调谐基线。代码可在https://github.com/xingchenwan/bgpbt上找到。

离线强化学习在利用大型预采用的数据集进行政策学习方面表现出了巨大的希望，使代理商可以放弃经常廉价的在线数据收集。但是，迄今为止，离线强化学习的探索相对较小，并且缺乏对剩余挑战所在的何处的了解。在本文中，我们试图建立简单的基线以在视觉域中连续控制。我们表明，对两个基于最先进的在线增强学习算法，Dreamerv2和DRQ-V2进行了简单的修改，足以超越事先工作并建立竞争性的基准。我们在现有的离线数据集中对这些算法进行了严格的评估，以及从视觉观察结果中进行离线强化学习的新测试台，更好地代表现实世界中离线增强学习问题中存在的数据分布，并开放我们的代码和数据以促进此方面的进度重要领域。最后，我们介绍并分析了来自视觉观察的离线RL所独有的几个关键Desiderata，包括视觉分散注意力和动态视觉上可识别的变化。

贝叶斯后期和模型证据的计算通常需要数值整合。贝叶斯正交（BQ）是一种基于替代模型的数值整合方法，能够具有出色的样品效率，但其缺乏并行化阻碍了其实际应用。在这项工作中，我们提出了一种并行的（批次）BQ方法，该方法采用了核正素的技术，该技术具有证明是指数的收敛速率。另外，与嵌套采样一样，我们的方法允许同时推断后期和模型证据。重新选择了来自BQ替代模型的样品，通过内核重组算法获得一组稀疏的样品，需要可忽略的额外时间来增加批处理大小。从经验上讲，我们发现我们的方法显着优于在包括锂离子电池分析在内的各种现实世界数据集中，最先进的BQ技术和嵌套采样的采样效率。

图表神经网络，一种流行的模型，在各种基于图形的学习任务中有效，已被证明易受对抗攻击的影响。虽然大多数文献侧重于节点级分类任务中的这种脆弱性，但很少努力致力于分析对图形级分类的对抗攻击，这是生物化学和社会网络分析等众多现实生活应用的重要问题。少数现有方法通常需要不切实际的设置，例如访问受害者模型的内部信息，或者是一个不切实际的查询。我们提出了一种新型贝叶斯优化的攻击方法，用于图形分类模型。我们的方法是黑匣子，查询效率和涉及扰动的效率和解析。我们经验验证了所提出的方法对涉及不同图形属性，约束和攻击方式的图形分类任务的效果和灵活性。最后，我们分析了产生的对手样本后面的常见可解释模式，这可能会在图形分类模型的对抗鲁棒性上流出进一步的光。

We present a dynamic path planning algorithm to navigate an amphibious rotor craft through a concave time-invariant obstacle field while attempting to minimize energy usage. We create a nonlinear quaternion state model that represents the rotor craft dynamics above and below the water. The 6 degree of freedom dynamics used within a layered architecture to generate motion paths for the vehicle to follow and the required control inputs. The rotor craft has a 3 dimensional map of its surroundings that is updated via limited range onboard sensor readings within the current medium (air or water). Path planning is done via PRM and D* Lite.

Logic Mill is a scalable and openly accessible software system that identifies semantically similar documents within either one domain-specific corpus or multi-domain corpora. It uses advanced Natural Language Processing (NLP) techniques to generate numerical representations of documents. Currently it leverages a large pre-trained language model to generate these document representations. The system focuses on scientific publications and patent documents and contains more than 200 million documents. It is easily accessible via a simple Application Programming Interface (API) or via a web interface. Moreover, it is continuously being updated and can be extended to text corpora from other domains. We see this system as a general-purpose tool for future research applications in the social sciences and other domains.

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world, and early DR detection is necessary to prevent vision loss and support an appropriate treatment. In this work, we leverage interactive machine learning and introduce a joint learning framework, termed DRG-Net, to effectively learn both disease grading and multi-lesion segmentation. Our DRG-Net consists of two modules: (i) DRG-AI-System to classify DR Grading, localize lesion areas, and provide visual explanations; (ii) DRG-Expert-Interaction to receive feedback from user-expert and improve the DRG-AI-System. To deal with sparse data, we utilize transfer learning mechanisms to extract invariant feature representations by using Wasserstein distance and adversarial learning-based entropy minimization. Besides, we propose a novel attention strategy at both low- and high-level features to automatically select the most significant lesion information and provide explainable properties. In terms of human interaction, we further develop DRG-Net as a tool that enables expert users to correct the system's predictions, which may then be used to update the system as a whole. Moreover, thanks to the attention mechanism and loss functions constraint between lesion features and classification features, our approach can be robust given a certain level of noise in the feedback of users. We have benchmarked DRG-Net on the two largest DR datasets, i.e., IDRID and FGADR, and compared it to various state-of-the-art deep learning networks. In addition to outperforming other SOTA approaches, DRG-Net is effectively updated using user feedback, even in a weakly-supervised manner.

Recent advances in upper limb prostheses have led to significant improvements in the number of movements provided by the robotic limb. However, the method for controlling multiple degrees of freedom via user-generated signals remains challenging. To address this issue, various machine learning controllers have been developed to better predict movement intent. As these controllers become more intelligent and take on more autonomy in the system, the traditional approach of representing the human-machine interface as a human controlling a tool becomes limiting. One possible approach to improve the understanding of these interfaces is to model them as collaborative, multi-agent systems through the lens of joint action. The field of joint action has been commonly applied to two human partners who are trying to work jointly together to achieve a task, such as singing or moving a table together, by effecting coordinated change in their shared environment. In this work, we compare different prosthesis controllers (proportional electromyography with sequential switching, pattern recognition, and adaptive switching) in terms of how they present the hallmarks of joint action. The results of the comparison lead to a new perspective for understanding how existing myoelectric systems relate to each other, along with recommendations for how to improve these systems by increasing the collaborative communication between each partner.